dask array operations

Offifial documents

Tips

Dask Arrays — How to Parallelize Numpy With Ease | by Dario Radečić | Towards Data Science

Pangeo use case - when dask.array and xarray.apply_ufunc are not the answer · Issue '#143 · pangeo-data/pangeo · GitHub

Examples

code:python

from scipy.stats import linregress

def calc_trend(data):

n=len(data)

slope, intercept, r_value, p_value, std_err = linregress(np.arange(n), data)

return slope*100.,p_value

from dask.array import apply_along_axis

results_delayed=apply_along_axis(calc_trend,0,da)

result=results_delayed.compute()

python xarray - dask performance apply along axis - Stack Overflow

python - dask.array.apply_along_axis: using each row of dask.array as an input to another function fails because of additional element (1) - Stack Overflow

code:python

da.apply_along_axis(arr=darr, axis=1, func1d=da.sum).compute()

with Progress bar

code:python

from dask.diagnostics import ProgressBar

with ProgressBar():

compute(result_delayed)